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Bi-Ber Bilderkennungssysteme Berlin 
Muggelseedamm 221, 12587 Berlin 

Method and apparatus for automated detection and checking of geometrical 
and/or textural features of an object 
10 " 

The invention concerns a method as set forth in the classifying 
portion of ciaim i and an apparatus for carrying out the method. 

The invention lies in the technical field of optically checking various 
views of an object, which is to be carried into effect for example in the 
15 process of circuit production and assembly. 

In regard to circuit production and assembly it is particularly 
necessary to check the prints on the top side of circuits in terms of their 
existence there and their quality (discernibility) in order to prevent 
confusions from occurring. In addition it is necessary to measure the 
20 connections ('terminal or connection legs') which are disposed at the 
underside of the circuits, to ascertain whether they all lie within a 
predetermined tolerance range in one plane so that for example in a 
subsequent soldering process they can be simultaneously and reliably 
involved with the solder. That checking operation or ascertaining the 
25 deviation of the ends of the terminal legs from a flat contact surface is 
referred to as coplanarity checking. This coplanarity checking operation can 
be effected from a side view on to the rows of terminal legs at the sides of 
the circuit. Besides the above-mentioned print quality assessment and the 
coplanarity checking, the housing bottom clearance of a component and 
30 pitch, twist or skew, width, position, length deviation and contact point of 
the terminal connections are measured in the context of a so-called "lead 
and mark inspection'. 



1 



Automatic reading of plain text on products is state of the art; see 
for example R Koy-Oberthur: 'Ubersicht industrieller Anwendungen der 
Klarschrift- und Barcode-identifikation', Symposium: Aktuelle 
Entwicklungen und Realisierungen der Bildverarbietung, 11th and 12th 
September 1997, Aachen, Ministerium fur Wirtschaft, Mittelstand und 
Technologie des Landes Nordrhein-Westfalen. The aim pursued by that 
procedure is to recognise the individual symbols contained in the script, but 
not to derive an assessing measurement in terms of the readability 
afforded in any way thereby. It is precisely this however that is necessary 
and appropriate for such situations if the content of the printing is known 
per se but its potential readability has to be guaranteed, for example for 
liability and monitoring situations. 

The lecture by T Schroeter: 'Einsatz der Bildverarbeitung zur 
Druckvollstandigkeitskontrolle' at the same Symposium described a method 
which detects the completeness of printing, that is to say checks it for 
missing characters. That method makes use of the so-called histogram 
information. The method described herein assesses the printed area 
without deriving an evaluation in terms of the readability of the 
identification. 

It is also known to implement a coplanarity check on circuits by 
means of optical methods with which gap widths can be measured. In the 
procedure described by Qtec in 'Industrielle Bildverarbeitung/Maschine 
Vision', VDM, Robotik Automation, Maschinenbau Verlag GmbH, 
Frankfurt/Main. 1997, an image is taken from the underside of the circuit to 
be checked, that image permitting a view on to shadows cast by the rows 
of connection legs. A disadvantage in this respect is that evaluation is 
effected only from one image which is recorded directly from an individual 
side of the object to be monitored, and thus it is not possible also to 
implement a print quality check. The location of the shadow of a connection 
leg to be measured, on a base surface, is moreover influenced not only by 
the distance thereof from the base surface (on which the shadow is cast) 
but also the location of the connection leg over that base surface, and 
moreover the specific shape of the connection leg, from which major errors 



can result. Furthermore, with this procedure, a plurality of individual light 
sources are required to produce an evaluatable shadow image. 

The coplanarity monitoring unit SMD9000 from Trigon Adcotech uses 
five CCD-cameras for recording the images to be evaluated, for monitoring 
coplanarity, one for each side view and one for a plan view. A disadvantage 
in this respect is the high level of technical complication and expenditure 
arising out of the number of cameras required, and the fact that the 
individual views are not linked to each other. 

In addition WO 98/04882 discloses a method of the general kind set 
forth for the automated detection and checking of geometrical and textural 
features of an IC component in side views and a plan view. In that case 
detection of the features is effected using a CCD-camera and a 
corresponding storage unit and evaluation unit for image processing and 
image evaluation. In that case, quality and state assessment of the object 
is effected by a comparison with parameters which are predetermined in 
respect of the individual features, wherein the partial images of the object 
are substantially simultaneously recorded by means of a single CCD- 
camera by virtue of a suitable beam-deflection device and are at least 
partially assembled optically at the same time to form an overall image 
which shows all views and in which the boundaries of the partial images can 
be recognised. The overall image is then separately evaluated for checking 
the individual features in the boundaries of the partial images. In this case 
the coplanarity of the legs of the IC-component is also checked. That 
publication however does not disclose the form in which that evaluation 
procedure is implemented. 

Therefore the object of the present invention is to provide a method 
and an apparatus which make it possible at relatively low cost to detect and 
check in interrelated manner and reliably in relation to coplanarity 
structural and/or textural features of an object in a plurality of views, more 
especially in a plan view and in side views. 

In regard to its method aspect that object is attained by a method 
having the features of claim 1 and in regard to its apparatus aspect by an 
apparatus having the features recited in claim 7. 
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The invention involves the technical teaching of obtaining from a 
single viewing position by means of suitable optical means an overall image 
constructed from partial images of various views of the object with a level 
of resolution which is sufficient for simultaneous structure and texture 
5 checking. 

The partial images are substantially simultaneously recorded and at 
least partially brought together optically at the same time to form an 
overall image showing all views, in which the boundaries of the partial 
images can be seen, and that is evaluated in the boundaries of the partial 

10 images separately - that is to say with different evaluation means or in 
terms of different parameters. 

In accordance with the invention firstly in the regions of the overall 
image which show side views on to the object, analysis of the gray value 
distributions is effected to ascertain locations in which parts (to be checked 

15 in terms of coplanarity) of the object come very close to a substrate. 
Subsequently the light quantity which passes through between the object 
and the substrate and which is reflected at the pixels as an intensity value 
is detected and, utilising the intensity values, the local light quantity 
pattern or configuration which characterises the width of a gap between the 

20 object and the substrate is determined. Finally the light quantity pattern or 
configuration can be converted into a gap width in accordance with a 
predetermined algorithm, utilising calibration information. The rule for 
conversion of an intensity pattern as between object and support is 
advantageously implemented on the basis of a spline approximation, the 

25 determination of rise values and calibration, in such a way that an area 
proportion under the spline curve determines the value of the gap width 
between the object and the support surface. That procedure is expedient 
especially for coplanarity checking of circuits or similar objects. 

In a preferred embodiment all partial images are assembled optically 

30 and recorded by precisely one image-recording device, wherein in the 
overall image the regions of the partial images are so positioned and 
characterised, on particular using the storage unit and/or the evaluation 
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unit, in such a way that they can be associated for the evaluation 
procedure with the individual views. 

Checking the object is made easier in many uses if in at least one 
additional step the scene is recorded without an object and/or with a 
5 reference object which has predetermined parameters in regard to the 
features, and the corresponding overall image is stored in the storage 
device for comparison and calibration purposes. 

In a region of the overall image (which for example shows the plan 
view of a circuit), for the purposes of detecting a texture and especially for 
10 checking the readability of a characterisation or identification, preferably by 
means of image processing, using convolution filters, areas with severe 
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of their dimensions, the result of the quantification operation in the above- 
mentioned areas is compared to given values and subsequently quality 

15 information is derived in respect of the texture or identification. 

In the apparatus according to the invention there is provided a flat 
support surface for the object, and the beam-deflection means are 
substantially arranged in the plane of the support surface in such a way 
that there is a view parallel to the support surface, permitting checking of 

20 the coplanarity of a plurality of parts of the object, which are towards the 
support surface. 

The apparatus according to the invention preferably includes a single 
image-recording device, relative to which the object is positioned in such a 
way that it fills only a part of its field of view, wherein arranged in 

25 remaining parts of the field of view are beam-deflection devices which 
produce images of side views of the object on the image-recording device. 

The beam-deflection devices are so positioned that they respectively 
produce an image of a given view of the object, but do not influence the 
other views (especially the plan view on to the object). 

30 The means for beam deflection have in particular prisms or mirrors 

which are fixed or displaceable and which can have surfaces which are 
optionally curved for specific uses. In addition or alternatively thereto they 
may have a light guide arrangement (one or more glass fiber bundles). 
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Associated with at least one of the .beam-deflection devices are 
means for altering the imaging scale of at least one partial image with 
respect to at least one other partial image, in particular a lens 
arrangement. 

5 Furthermore the apparatus preferably includes - especially for 

coplanarity checking procedures - a lighting device which in particular has a 
light diffuser device for producing a regular light flux under the object, 
which is arranged behind (from the point of view of the beam-deflection 
device) projecting parts of the object. 
10 In that case the light diffuser device can preferably be interrupted in 

such a way that the free spaces permit a view on to the underside of the 

w is J V— V_ 1- . 

For recording a plan view, in a preferred feature a further lighting 
device which is suitable specifically for evaluating the recognisability of a 
15 surface texture is then arranged over the object. 

A preferred apparatus is one in which the image-recording device 
and the storage and evaluation unit are integrated in a structural unit. That 
can be for example a so-called 'smart camera 1 , as however is hitherto not 
used for lead and mark inspection. The advantages of using an integrated 
20 component are that on the one hand it is easily obtainable and on the other 
hand it can be easily configured for use in the context of the procedures 
described hereinbefore. 

Advantageous embodiments of the invention are also set forth in the 
appendant claims, and the description hereinafter with reference to the 
25 accompanying drawings in which: 

Figure 1 shows a diagrammatic simplified view of the overall 
structure of an embodiment of an apparatus according to the invention, 

Figure 2 shows an overall image of an example of an object to be 
checked, with a plan view and side views recorded from a viewing position, 
30 Figure 3 is a view of part of a further embodiment with 

advantageously arranged lighting devices for producing top light and 
relative transillumination, 
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Figure 4 shows a view of the intensity profile of a gap section 
between a support and a projecting part of an object, which is towards 
same, 

Figure 5 shows a side view and a view from below of a component 
5 with L-shaped connections, and a side view and a view from below of a 
component with J-shaped connections. 

Figure 1 is a diagrammatic view showing the principle of the overall 
structure of an apparatus for checking features, which can be recognised in 
side views and a plan view, of an object 1, from a single viewing position. 
10 The apparatus includes a CCD-camera 2 as an image-recording device, with 
an objective lens 2a which has a field of view 3, a support surface 4, prisms 
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of the deflected radiation, as well as an image evaluation and storage unit 
7. A lighting device is not shown; the apparatus by way of example 

15 accordingly operates with diffuse ambient light. 

The camera 2 records an overall image of the object 1, which is 
composed of a plurality of partial images of various views of the object. The 
light which is reflected from the top side la of the object and which passes 
directly into the objective lens 2a produces a partial image of the plan view, 

20 and the light which is reflected from the side surfaces lb and which is 
deflected by the prisms 5 into the objective lens of the camera produces 
partial images corresponding to the side views. The lenses 6 produce a 
change in the imaging scale or the detailing of the recorded image in the 
parts of the beam path which are influenced by the lenses, that is to say in 

25 the partial images of the side views. 

The camera 2 transmits the overall image in the form of an electrical 
signal pattern to the image evaluation and storage device 7 (not described 
in greater detail herein) which for example can be embodied by a personal 
computer. The image evaluation and storage unit 7 determines the 

30 boundaries and demarcation of the partial images and, using image 
processing and image evaluation methods which are known per se but 
which are specifically adapted to the features to be checked in the partial 
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images, extracts relevant items of information and parameters from the 
partial images, and prepares them for comparative evaluation. 

Figure 2 shows a typical overall image 11 of a circuit in the field of 
view 10 of the camera, the image having been recorded by the camera 2 in. 
5 the apparatus shown in Figure 1 and stored in the image evaluation and 
storage unit 7. The overall image 10 comprises five partial images, more 
specifically the partial image 11a of the plan view and the partial images 
lib through lie of the side views. Printing 12 on the top side of the circuit 
stands out clearly in its intensity from the intensity of the ambient light. 

10 The images of the side views are formatively influenced by superposition 
structures 13 which include elements which are caused by gaps of different 
widths between the legs of the circuit 1 and the support 4. The 
superposition structures are evaluated by means of special image 
processing methods as already referred to hereinbefore and used for a 

15 checking procedure in respect of coplanarity of the circuit connections. 

Figure 3 shows a detail of a modified embodiment of the 
arrangement shown in Figure 1, in which, in addition to the components 
illustrated therein, there are provided two lighting devices 8a, 8b for 
producing top light on the surface of the object 1 and for producing 

20 transmission light for the side views of the legs 1.1 of the circuit 1, the 
transmission light being rendered diffuse by a light diffuser device 9. 

Figure 4 shows a spline curve S for the variation in intensity in the 
pixels of the digitised partial image of a side view for one of the gaps 
between a leg and the support. The area A s under the spline curve S, 

25 defined by the rise points X 0 and Xi, is used in the context of a specific 
image evaluation procedure for the side view partial image for computation 
of the gap width as a coplanarity parameter. 

Figure 5 denotes the parameters checked in the context of a so- 
called lead and mark inspection, by reference to the side view lb of an 

30 object 1 and by reference to the underneath view lc thereof. The object 
shown at the left in Figure 5 has L-shaped connections while the object 
shown at the right in Figure 5 has J-shaped connections. The parameters to 
be checked are the coplanarity CP, the housing bottom clearance ST, the 
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pitch PI, the twist or skew SK, the connection or lead width LW, the length 
LS, the connection or lead position LP, the length deviation LED, the 
connection dimensions TD, the contact points or the footprint FP and the 
housing height (not identified). 
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